Prodding the ROC curve: Constrained optimization of classifier performance
When designing a two-alternative classifier, one ordinarily aims to maximize
the classifier's ability to discriminate between members of the two classes.
We describe a situation in a real-world business application of
machine-learning prediction in which an additional constraint is placed on the
nature of the solution: that the classifier achieve a specified correct
acceptance or correct rejection rate (i.e., that it achieve a fixed accuracy
on members of one class or the other). Our domain is predicting churn in the
telecommunications industry. Churn refers to customers who switch from one
service provider to another. We propose four algorithms for training a
classifier subject to this domain constraint, and present results showing that
each algorithm yields a reliable improvement in performance. Although the
improvement is modest in magnitude, it is nonethe less impressive given the
difficulty of the problem and the financial return that it achieves to the
service provider.
Retrieve Paper (postscript)
Retrieve Paper (pdf)